

<!DOCTYPE html>
<html lang="zh-CN" data-default-color-scheme=&#34;auto&#34;>



<head>
  <meta charset="UTF-8">
  <link rel="apple-touch-icon" sizes="76x76" href="/img/2.jpg">
  <link rel="icon" type="image/png" href="/img/2.jpg">
  <meta name="viewport"
        content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no, shrink-to-fit=no">
  <meta http-equiv="x-ua-compatible" content="ie=edge">
  
  <meta name="theme-color" content="#2f4154">
  <meta name="description" content="">
  <meta name="author" content="K">
  <meta name="keywords" content="">
  <title>docker-flannel - K</title>

  <link  rel="stylesheet" href="https://cdn.staticfile.org/twitter-bootstrap/4.4.1/css/bootstrap.min.css" />


  <link  rel="stylesheet" href="https://cdn.staticfile.org/github-markdown-css/4.0.0/github-markdown.min.css" />
  <link  rel="stylesheet" href="/lib/hint/hint.min.css" />

  
    
    
      
      <link  rel="stylesheet" href="https://cdn.staticfile.org/highlight.js/10.0.0/styles/github-gist.min.css" />
    
  

  


<!-- 主题依赖的图标库，不要自行修改 -->

<link rel="stylesheet" href="//at.alicdn.com/t/font_1749284_ba1fz6golrf.css">



<link rel="stylesheet" href="//at.alicdn.com/t/font_1736178_kmeydafke9r.css">


<link  rel="stylesheet" href="/css/main.css" />

<!-- 自定义样式保持在最底部 -->


  <script  src="/js/utils.js" ></script>
  <script  src="/js/color-schema.js" ></script>
<meta name="generator" content="Hexo 5.2.0"></head>


<body>
  <header style="height: 70vh;">
    <nav id="navbar" class="navbar fixed-top  navbar-expand-lg navbar-dark scrolling-navbar">
  <div class="container">
    <a class="navbar-brand"
       href="/">&nbsp;<strong>かい</strong>&nbsp;</a>

    <button id="navbar-toggler-btn" class="navbar-toggler" type="button" data-toggle="collapse"
            data-target="#navbarSupportedContent"
            aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
      <div class="animated-icon"><span></span><span></span><span></span></div>
    </button>

    <!-- Collapsible content -->
    <div class="collapse navbar-collapse" id="navbarSupportedContent">
      <ul class="navbar-nav ml-auto text-center">
        
          
          
          
          
            <li class="nav-item">
              <a class="nav-link" href="/">
                <i class="iconfont icon-home-fill"></i>
                首页
              </a>
            </li>
          
        
          
          
          
          
            <li class="nav-item">
              <a class="nav-link" href="/archives/">
                <i class="iconfont icon-archive-fill"></i>
                归档
              </a>
            </li>
          
        
          
          
          
          
            <li class="nav-item">
              <a class="nav-link" href="/categories/">
                <i class="iconfont icon-category-fill"></i>
                分类
              </a>
            </li>
          
        
          
          
          
          
            <li class="nav-item">
              <a class="nav-link" href="/tags/">
                <i class="iconfont icon-tags-fill"></i>
                标签
              </a>
            </li>
          
        
          
          
          
          
            <li class="nav-item">
              <a class="nav-link" href="/about/">
                <i class="iconfont icon-user-fill"></i>
                关于
              </a>
            </li>
          
        
        
          <li class="nav-item" id="search-btn">
            <a class="nav-link" data-toggle="modal" data-target="#modalSearch">&nbsp;<i
                class="iconfont icon-search"></i>&nbsp;</a>
          </li>
        
        
          <li class="nav-item" id="color-toggle-btn">
            <a class="nav-link" href="javascript:">&nbsp;<i
                class="iconfont icon-dark" id="color-toggle-icon"></i>&nbsp;</a>
          </li>
        
      </ul>
    </div>
  </div>
</nav>

    <div class="banner intro-2" id="background" parallax=true
         style="background: url('/img/1.jpg') no-repeat center center;
           background-size: cover;">
      <div class="full-bg-img">
        <div class="mask flex-center" style="background-color: rgba(0, 0, 0, 0.3)">
          <div class="container page-header text-center fade-in-up">
            <span class="h2" id="subtitle">
              
            </span>

            
              <div class="mt-3">
  
  
    <span class="post-meta">
      <i class="iconfont icon-date-fill" aria-hidden="true"></i>
      <time datetime="2020-12-11 14:00" pubdate>
        2020年12月11日 下午
      </time>
    </span>
  
</div>

<div class="mt-1">
  
    
    <span class="post-meta mr-2">
      <i class="iconfont icon-chart"></i>
      3.9k 字
    </span>
  

  
    
    <span class="post-meta mr-2">
      <i class="iconfont icon-clock-fill"></i>
      
      
      60
       分钟
    </span>
  

  
  
</div>

            
          </div>

          
        </div>
      </div>
    </div>
  </header>

  <main>
    
      

<div class="container-fluid">
  <div class="row">
    <div class="d-none d-lg-block col-lg-2"></div>
    <div class="col-lg-8 nopadding-md">
      <div class="container nopadding-md" id="board-ctn">
        <div class="py-5" id="board">
          <article class="post-content mx-auto" id="post">
            <!-- SEO header -->
            <h1 style="display: none">docker-flannel</h1>
            
            <div class="markdown-body" id="post-body">
              <h2 id="flannel"><a href="#flannel" class="headerlink" title="flannel"></a>flannel</h2><ul>
<li>flannel 为每个主机分配一个(subnet)子网，容器从这个子网中分配ip ，这些ip可以在主机之间进行路由</li>
<li>每个容器之间不需要nat和端口映射，就能完成跨主机通讯</li>
<li>通过计算机的最长匹配原则来进行的</li>
</ul>
<h2 id="flannel数据转发"><a href="#flannel数据转发" class="headerlink" title="flannel数据转发"></a>flannel数据转发</h2><ul>
<li>数据会通过 backend来从etcd进行转发</li>
<li>数据包如何在主机间转发是由 backend 实现的。flannel 提供了多种 backend，最常用的有 vxlan 和 host-gw</li>
<li>其他 backend参数 请参考 <a target="_blank" rel="noopener" href="https://github.com/coreos/flannel">https://github.com/coreos/flannel</a></li>
</ul>
<h2 id="flannel转发原理"><a href="#flannel转发原理" class="headerlink" title="flannel转发原理"></a>flannel转发原理</h2><p>  <img src="/images/docker/flannel.png" srcset="/img/loading.gif"></p>
<ul>
<li><p>创建好flannel网络时，用户指定一个大的网段通过上传到etcd，etcd通过backend来转发给加入etcd集群的主机，flannel收到后，会根据这个大的网段创建容器，将docker0修改为大网段的子网段，还会创建一个flannel.1网卡，用来进行udp封装(选择udp作为转发协议，是因为udp能穿透防火墙)</p>
</li>
<li><p>vxlan模式下，容器发送数据到别的主机的容器时，会经过docker0然后转发到flannel1网卡，通过p2p的udp封装，按路由(最长匹配原则)，找到与之最接近的路由，转发到那一台主机上，到另一台主机上进行相反操作(udp解封装，传给docker0，到容器)</p>
</li>
<li><p>host-gw模式下，容器会通过etcd的信息数据创建到对端主机的路由条目，容器发送数据，会经过docker0,也就是docker0作为子网的网关，然后，通过本机路由表找到到对端主机的路由，直接走路由通信，不会经过flannel进行udp封装</p>
</li>
</ul>
<p>   <img src="/images/docker/flannlezf.png" srcset="/img/loading.gif"></p>
<blockquote>
<p>根据上图演示</p>
</blockquote>
<ul>
<li><p>web1发送消息到web2会先通过经过docker0，如果主机的vxlan模式会走flannel.1经过最长匹配原则，匹配最接近web2的ip地址段，通过upd封装发送到该地址段，对端主机经过flannel解包，然后通过docker0传给web2</p>
</li>
<li><p>web1发送消息到web2会先通过经过docker0 如果主机是host_gw模式,会走物理主机的路由条目，找到最匹配的物理机的路由条目，然后发送信息到该网段，对端主机操作相反，收到信息</p>
</li>
</ul>
<h2 id="配置flannel"><a href="#配置flannel" class="headerlink" title="配置flannel"></a>配置flannel</h2><table>
<thead>
<tr>
<th>192.168.100.211</th>
<th>docker etcd flanneld</th>
</tr>
</thead>
<tbody><tr>
<td>192.168.100.212</td>
<td>docker etcd flanneld</td>
</tr>
</tbody></table>
<h2 id="准备环境"><a href="#准备环境" class="headerlink" title="准备环境"></a>准备环境</h2><ul>
<li>使用flannel关闭防火墙，selinux 开启路由转发</li>
<li>如果不关防火墙，放行2380/2379/4001三个端口<pre><code class="hljs perl">[root@localhost ~]<span class="hljs-comment"># systemctl disable firewalld</span>
Removed <span class="hljs-keyword">symlink</span> /etc/systemd/<span class="hljs-keyword">system</span>/multi-user.target.wants/firewalld.service.
Removed <span class="hljs-keyword">symlink</span> /etc/systemd/<span class="hljs-keyword">system</span>/dbus-org.fedoraproject.FirewallD1.service.
[root@localhost ~]<span class="hljs-comment"># systemctl stop firewalld</span>
[root@localhost ~]<span class="hljs-comment"># vim /etc/selinux/config </span>
SELINUX=disabled
[root@localhost ~]<span class="hljs-comment"># setenforce 0</span>
[root@localhost ~]<span class="hljs-comment"># getenforce </span>
Permissive</code></pre>
<blockquote>
<p>清除防火墙策略</p>
</blockquote>
<pre><code class="hljs autoit">[root<span class="hljs-symbol">@localhost</span> ~]<span class="hljs-meta"># iptables -P FORWARD ACCEPT</span>
</code></pre>
<blockquote>
<p>安装 flannel 和 etcd</p>
</blockquote>
<pre><code class="hljs autoit">[root<span class="hljs-symbol">@localhost</span> ~]<span class="hljs-meta"># yum -y install flannel etcd</span>
</code></pre>
<blockquote>
<p>配置etcd</p>
</blockquote>
<ul>
<li>192.168.100.211</li>
<li>修改etcd配置文件，可以让etcd来识别两台docker主机，也可以说是集群，为他们去分配ip子网<pre><code class="hljs makefile">[root@localhost ~]<span class="hljs-comment"># cp -p /etc/etcd/etcd.conf /etc/etcd/etcd.conf.bak</span>
[root@localhost ~]<span class="hljs-comment"># vim /etc/etcd/etcd.conf</span>
<span class="hljs-comment"># etcd存放数据目录，为了方便改成了和etcd_name一样的名字</span>
ETCD_DATA_DIR=<span class="hljs-string">&quot;/var/lib/etcd/etcd1&quot;</span>
<span class="hljs-comment"># 用于与其他节点通信</span>
ETCD_LISTEN_PEER_URLS=<span class="hljs-string">&quot;http://192.168.100.211:2380&quot;</span>
<span class="hljs-comment"># 客户端会连接到这里和 etcd 交互</span>
ETCD_LISTEN_CLIENT_URLS=<span class="hljs-string">&quot;http://192.168.100.211:2379,http://127.0.0.1:2379&quot;</span>
<span class="hljs-comment"># 节点名称，每台主机都不一样</span>
ETCD_NAME=<span class="hljs-string">&quot;etcd1&quot;</span>
该节点同伴监听地址，这个值会告诉集群中其他节点
ETCD_INITIAL_ADVERTISE_PEER_URLS=<span class="hljs-string">&quot;http://192.168.100.211:2380&quot;</span>
<span class="hljs-comment"># 对外公告的该节点客户端监听地址，这个值会告诉集群中其他节点</span>
ETCD_ADVERTISE_CLIENT_URLS=<span class="hljs-string">&quot;http://192.168.100.211:2379&quot;</span>
<span class="hljs-comment"># 集群中的主机信息</span>
ETCD_INITIAL_CLUSTER=<span class="hljs-string">&quot;etcd1=http://192.168.100.211:2380,etcd2=http://192.168.100.212:2380&quot;</span>
<span class="hljs-comment"># 集群token，建议修改一个，同一个集群中的token一致</span>
ETCD_INITIAL_CLUSTER_TOKEN=<span class="hljs-string">&quot;etcd-test&quot;</span>
<span class="hljs-comment"># 新建集群的时候，这个值为new假如已经存在的集群，这个值为 existing</span>
ETCD_INITIAL_CLUSTER_STATE=<span class="hljs-string">&quot;new&quot;</span></code></pre></li>
<li>192.168.100.212<pre><code class="hljs makefile">[root@localhost ~]<span class="hljs-comment"># cp -p /etc/etcd/etcd.conf /etc/etcd/etcd.conf.bak</span>
[root@localhost ~]<span class="hljs-comment"># vim /etc/etcd/etcd.conf</span>
<span class="hljs-comment"># etcd存放数据目录，为了方便改成了和etcd_name一样的名字</span>
ETCD_DATA_DIR=<span class="hljs-string">&quot;/var/lib/etcd/etcd2&quot;</span>
<span class="hljs-comment"># 用于与其他节点通信</span>
ETCD_LISTEN_PEER_URLS=<span class="hljs-string">&quot;http://192.168.100.212:2380&quot;</span>
<span class="hljs-comment"># 客户端会连接到这里和 etcd 交互</span>
ETCD_LISTEN_CLIENT_URLS=<span class="hljs-string">&quot;http://192.168.100.212:2379,http://127.0.0.1:2379&quot;</span>
<span class="hljs-comment"># 节点名称，每台主机都不一样</span>
ETCD_NAME=<span class="hljs-string">&quot;etcd2&quot;</span>
该节点同伴监听地址，这个值会告诉集群中其他节点
ETCD_INITIAL_ADVERTISE_PEER_URLS=<span class="hljs-string">&quot;http://192.168.100.212:2380&quot;</span>
<span class="hljs-comment"># 对外公告的该节点客户端监听地址，这个值会告诉集群中其他节点</span>
ETCD_ADVERTISE_CLIENT_URLS=<span class="hljs-string">&quot;http://192.168.100.212:2379&quot;</span>
<span class="hljs-comment"># 集群中的主机信息</span>
ETCD_INITIAL_CLUSTER=<span class="hljs-string">&quot;etcd1=http://192.168.100.211:2380,etcd2=http://192.168.100.212:2380&quot;</span>
<span class="hljs-comment"># 集群token，建议修改一个，同一个集群中的token一致</span>
ETCD_INITIAL_CLUSTER_TOKEN=<span class="hljs-string">&quot;etcd-test&quot;</span>
<span class="hljs-comment"># 新建集群的时候，这个值为new假如已经存在的集群，这个值为 existing</span>
ETCD_INITIAL_CLUSTER_STATE=<span class="hljs-string">&quot;new&quot;</span></code></pre>
<blockquote>
<p>etcd识别配置文件变量</p>
</blockquote>
</li>
<li>默认刚才修改的内容，都是etcd启动后不会识别的，所以需要在启动文件中添加<pre><code class="hljs lsl"># 两台主机都修改
[root@localhost ~]# vim /usr/lib/systemd/system/etcd.service 
# 在第十三行（ExecStart开头）末尾的前一个引号里面，添加如下
# 添加时不换行，空格隔开即可
--<span class="hljs-section">listen</span>-peer-urls=\<span class="hljs-string">&quot;$&#123;ETCD_LISTEN_PEER_URLS&#125;<span class="hljs-subst">\&quot;</span> </span>
<span class="hljs-string">--advertise-client-urls=<span class="hljs-subst">\&quot;</span>$&#123;ETCD_ADVERTISE_CLIENT_URLS&#125;<span class="hljs-subst">\&quot;</span> </span>
<span class="hljs-string">--initial-cluster-token=<span class="hljs-subst">\&quot;</span>$&#123;ETCD_INITIAL_CLUSTER_TOKEN&#125;<span class="hljs-subst">\&quot;</span> </span>
<span class="hljs-string">--initial-cluster=<span class="hljs-subst">\&quot;</span>$&#123;ETCD_INITIAL_CLUSTER&#125;<span class="hljs-subst">\&quot;</span> </span>
<span class="hljs-string">--initial-cluster-state=<span class="hljs-subst">\&quot;</span>$&#123;ETCD_INITIAL_CLUSTER_STATE&#125;<span class="hljs-subst">\&quot;</span></span>
<span class="hljs-string"></span>
<span class="hljs-string"># ExecStart=/bin/bash -c &quot;</span>GOMAXPROCS=$(nproc) /usr/bin/etcd --name=\<span class="hljs-string">&quot;$&#123;ETCD_NAME&#125;<span class="hljs-subst">\&quot;</span> --data-dir=<span class="hljs-subst">\&quot;</span>$&#123;ETCD_DATA_DIR&#125;<span class="hljs-subst">\&quot;</span> --listen-client-urls=<span class="hljs-subst">\&quot;</span>$&#123;ETCD_LISTEN_CLIENT_URLS&#125;<span class="hljs-subst">\&quot;</span> --listen-peer-urls=<span class="hljs-subst">\&quot;</span>$&#123;ETCD_LISTEN_PEER_URLS&#125;<span class="hljs-subst">\&quot;</span> --advertise-client-urls=<span class="hljs-subst">\&quot;</span>$&#123;ETCD_ADVERTISE_CLIENT_URLS&#125;<span class="hljs-subst">\&quot;</span> --initial-cluster=<span class="hljs-subst">\&quot;</span>$&#123;ETCD_INITIAL_CLUSTER&#125;<span class="hljs-subst">\&quot;</span> --initial-cluster-token=<span class="hljs-subst">\&quot;</span>$&#123;ETCD_INITIAL_CLUSTER_TOKEN&#125;<span class="hljs-subst">\&quot;</span> --initial-cluster-state=<span class="hljs-subst">\&quot;</span>$&#123;ETCD_INITIAL_CLUSTER_STATE&#125;<span class="hljs-subst">\&quot;</span>&quot;</span></code></pre>
<blockquote>
<p>启动etcd （两台）</p>
</blockquote>
<pre><code class="hljs autoit">[root<span class="hljs-symbol">@localhost</span> ~]<span class="hljs-meta"># systemctl daemon-reload</span>
[root<span class="hljs-symbol">@localhost</span> ~]<span class="hljs-meta"># systemctl start etcd</span>
</code></pre>
<blockquote>
<p>查看健康状态</p>
</blockquote>
<pre><code class="hljs angelscript"><span class="hljs-string">[root@localhost ~]</span># etcdctl cluster-health
member <span class="hljs-number">146f</span>6898c619f119 <span class="hljs-keyword">is</span> healthy: got healthy result <span class="hljs-keyword">from</span> http:<span class="hljs-comment">//192.168.100.211:2379</span>
member <span class="hljs-number">3f</span>00837914f85672 <span class="hljs-keyword">is</span> healthy: got healthy result <span class="hljs-keyword">from</span> http:<span class="hljs-comment">//192.168.100.212:2379</span>
cluster <span class="hljs-keyword">is</span> healthy

# 查看集群中的 leader是那台
<span class="hljs-string">[root@localhost ~]</span># etcdctl member list
<span class="hljs-number">146f</span>6898c619f119: name=etcd1 peerURLs=http:<span class="hljs-comment">//192.168.100.211:2380 clientURLs=http://192.168.100.211:2379 isLeader=true</span>
<span class="hljs-number">3f</span>00837914f85672: name=etcd2 peerURLs=http:<span class="hljs-comment">//192.168.100.212:2380 clientURLs=http://192.168.100.212:2379 isLeader=false</span>

# isLeader=<span class="hljs-literal">true</span> <span class="hljs-number">100.212</span>就是集群中的领导者，ip会从server1的flannel去分配
</code></pre>

</li>
</ul>
</li>
</ul>
<h2 id="flannel-vxlan类型容器互通"><a href="#flannel-vxlan类型容器互通" class="headerlink" title="flannel vxlan类型容器互通"></a>flannel vxlan类型容器互通</h2><ul>
<li><p>此类型的backend是通过UDP封装来实现的。</p>
</li>
<li><p>让flannel的etcd使用该文件中，定义的backend类型和给定的大网段范围10.10.0.0/16，然后使用24的子网掩码，在该范围中为容器自动划分ip</p>
<pre><code class="hljs autoit">[root<span class="hljs-symbol">@localhost</span> ~]<span class="hljs-meta"># vim /root/etcd.json</span>
&#123;
  <span class="hljs-string">&quot;NetWork&quot;</span>:<span class="hljs-string">&quot;10.10.0.0/16&quot;</span>,
  <span class="hljs-string">&quot;SubnetLen&quot;</span>:<span class="hljs-number">24</span>,
  <span class="hljs-string">&quot;Backend&quot;</span>:&#123;
    <span class="hljs-string">&quot;Type&quot;</span>:<span class="hljs-string">&quot;vxlan&quot;</span>
  &#125;
&#125;</code></pre>
<blockquote>
<p>将该文件导入etcd集群中</p>
</blockquote>
<ul>
<li>/usr/local/bin/network/config：该路径物理机并没有，是flannel的etcd内的<pre><code class="hljs routeros">[root@localhost ~]# etcdctl <span class="hljs-attribute">--endpoints</span>=http://192.168.100.211:2379  <span class="hljs-builtin-name">set</span> /usr/local/bin/network<span class="hljs-built_in">/config </span>&lt; /root/etcd.json
&#123;
  <span class="hljs-string">&quot;NetWork&quot;</span>:<span class="hljs-string">&quot;10.10.0.0/16&quot;</span>,
  <span class="hljs-string">&quot;SubnetLen&quot;</span>:24,
  <span class="hljs-string">&quot;Backend&quot;</span>:&#123;
    <span class="hljs-string">&quot;Type&quot;</span>:<span class="hljs-string">&quot;vxlan&quot;</span>
  &#125;
&#125;

<span class="hljs-comment"># etcd2 查看是否可以读取到 etcd集群中的 值</span>
[root@localhost ~]# etcdctl <span class="hljs-builtin-name">get</span> /usr/local/bin/network/config
&#123;
  <span class="hljs-string">&quot;NetWork&quot;</span>:<span class="hljs-string">&quot;10.10.0.0/16&quot;</span>,
  <span class="hljs-string">&quot;SubnetLen&quot;</span>:24,
  <span class="hljs-string">&quot;Backend&quot;</span>:&#123;
    <span class="hljs-string">&quot;Type&quot;</span>:<span class="hljs-string">&quot;vxlan&quot;</span>
  &#125;
&#125;
</code></pre>
<blockquote>
<p>更改flannel配置文件</p>
</blockquote>
</li>
<li>使得flannel可以识别这个文件<pre><code class="hljs ini"><span class="hljs-comment"># 192.168.100.211</span>
<span class="hljs-section">[root@localhost ~]</span><span class="hljs-comment"># vim /etc/sysconfig/flanneld</span>
<span class="hljs-attr">FLANNEL_ETCD_ENDPOINTS</span>=<span class="hljs-string">&quot;http://192.168.100.211:2379&quot;</span>
<span class="hljs-attr">FLANNEL_ETCD_PREFIX</span>=<span class="hljs-string">&quot;/usr/local/bin/network&quot;</span>
<span class="hljs-comment"># 192.168.100.212</span>
<span class="hljs-section">[root@localhost ~]</span><span class="hljs-comment"># vim /etc/sysconfig/flanneld</span>
<span class="hljs-attr">FLANNEL_ETCD_ENDPOINTS</span>=<span class="hljs-string">&quot;http://192.168.100.212:2379&quot;</span>
<span class="hljs-attr">FLANNEL_ETCD_PREFIX</span>=<span class="hljs-string">&quot;/usr/local/bin/network&quot;</span>
<span class="hljs-comment"># 两台主机启动服务</span>
<span class="hljs-section">[root@localhost ~]</span><span class="hljs-comment"># systemctl enable flanneld</span>
<span class="hljs-section">[root@localhost ~]</span><span class="hljs-comment"># systemctl start flanneld</span>
</code></pre>
<blockquote>
<p>查看flannel分配的ip</p>
</blockquote>
</li>
<li>查看启动flanneld后，随机在10.10.0.0/16地址池中分配的子网ip<pre><code class="hljs angelscript"><span class="hljs-string">[root@localhost ~]</span># ip a

# <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.211</span>
<span class="hljs-number">6</span>: flannel<span class="hljs-number">.1</span>: &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; mtu <span class="hljs-number">1450</span> qdisc noqueue state UNKNOWN group <span class="hljs-keyword">default</span> 
    link/ether <span class="hljs-number">0</span>e:bc:ce:<span class="hljs-number">0</span>b:<span class="hljs-number">12</span>:<span class="hljs-number">1f</span> brd ff:ff:ff:ff:ff:ff
    inet <span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.0</span>/<span class="hljs-number">32</span> scope global flannel<span class="hljs-number">.1</span>
       valid_lft forever preferred_lft forever
    inet6 fe80::cbc:ceff:fe0b:<span class="hljs-number">121f</span>/<span class="hljs-number">64</span> scope link 
       valid_lft forever preferred_lft forever


#  <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.212</span>
<span class="hljs-number">6</span>: flannel<span class="hljs-number">.1</span>: &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; mtu <span class="hljs-number">1450</span> qdisc noqueue state UNKNOWN group <span class="hljs-keyword">default</span> 
    link/ether fe:<span class="hljs-number">78</span>:<span class="hljs-number">71</span>:<span class="hljs-number">7</span>b:<span class="hljs-number">17</span>:ee brd ff:ff:ff:ff:ff:ff
    inet <span class="hljs-number">10.10</span><span class="hljs-number">.88</span><span class="hljs-number">.0</span>/<span class="hljs-number">32</span> scope global flannel<span class="hljs-number">.1</span>
       valid_lft forever preferred_lft forever
    inet6 fe80::fc78:<span class="hljs-number">71f</span>f:fe7b:<span class="hljs-number">17</span>ee/<span class="hljs-number">64</span> scope link 
       valid_lft forever preferred_lft forever
</code></pre>

</li>
</ul>
<blockquote>
<p>docker0 连接 flannel.1</p>
</blockquote>
<ul>
<li>查看子网的MTU值与flannel子网网关<pre><code class="hljs angelscript"># <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.211</span>
<span class="hljs-string">[root@localhost ~]</span># cat /run/flannel/subnet.env 
FLANNEL_NETWORK=<span class="hljs-number">10.10</span><span class="hljs-number">.0</span><span class="hljs-number">.0</span>/<span class="hljs-number">16</span>
FLANNEL_SUBNET=<span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.1</span>/<span class="hljs-number">24</span>
FLANNEL_MTU=<span class="hljs-number">1450</span>
FLANNEL_IPMASQ=<span class="hljs-literal">false</span>

# <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.212</span>
<span class="hljs-string">[root@localhost ~]</span># cat /run/flannel/subnet.env 
FLANNEL_NETWORK=<span class="hljs-number">10.10</span><span class="hljs-number">.0</span><span class="hljs-number">.0</span>/<span class="hljs-number">16</span>
FLANNEL_SUBNET=<span class="hljs-number">10.10</span><span class="hljs-number">.88</span><span class="hljs-number">.1</span>/<span class="hljs-number">24</span>
FLANNEL_MTU=<span class="hljs-number">1450</span>
FLANNEL_IPMASQ=<span class="hljs-literal">false</span>
</code></pre>
<blockquote>
<p>修改docker启动项</p>
</blockquote>
<pre><code class="hljs awk"><span class="hljs-comment"># 192.168.100.211</span>
[root@localhost ~]<span class="hljs-comment"># vim /usr/lib/systemd/system/docker.service</span>
<span class="hljs-comment"># 第十四行</span>
ExecStart=<span class="hljs-regexp">/usr/</span>bin<span class="hljs-regexp">/dockerd -H fd:/</span><span class="hljs-regexp">/ --containerd=/</span>run<span class="hljs-regexp">/containerd/</span>containerd.sock --bip=<span class="hljs-number">10.10</span>.<span class="hljs-number">85.1</span>/<span class="hljs-number">24</span> --mtu=<span class="hljs-number">1450</span>

<span class="hljs-comment"># 192.168.100.212</span>
[root@localhost ~]<span class="hljs-comment"># vim /usr/lib/systemd/system/docker.service</span>
<span class="hljs-comment"># 第十四行</span>
ExecStart=<span class="hljs-regexp">/usr/</span>bin<span class="hljs-regexp">/dockerd -H fd:/</span><span class="hljs-regexp">/ --containerd=/</span>run<span class="hljs-regexp">/containerd/</span>containerd.sock --bip=
<span class="hljs-number">10.10</span>.<span class="hljs-number">88.1</span>/<span class="hljs-number">24</span> --mtu=<span class="hljs-number">1450</span>

<span class="hljs-comment"># 重启docker</span>
[root@localhost ~]<span class="hljs-comment"># systemctl daemon-reload </span>
[root@localhost ~]<span class="hljs-comment"># systemctl restart docker</span></code></pre>
<blockquote>
<p>查看docker0 是否变为 10网段的了</p>
</blockquote>
<pre><code class="hljs angelscript"><span class="hljs-string">[root@localhost ~]</span># ip a
# <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.211</span>
<span class="hljs-number">5</span>: docker0: &lt;NO-CARRIER,BROADCAST,MULTICAST,UP&gt; mtu <span class="hljs-number">1500</span> qdisc noqueue state DOWN group <span class="hljs-keyword">default</span> 
    link/ether <span class="hljs-number">02</span>:<span class="hljs-number">42</span>:d0:<span class="hljs-number">82</span>:<span class="hljs-number">52</span>:<span class="hljs-number">93</span> brd ff:ff:ff:ff:ff:ff
    inet <span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.1</span>/<span class="hljs-number">24</span> brd <span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.255</span> scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::<span class="hljs-number">42</span>:d0ff:fe82:<span class="hljs-number">5293</span>/<span class="hljs-number">64</span> scope link 
       valid_lft forever preferred_lft forever
<span class="hljs-number">6</span>: flannel<span class="hljs-number">.1</span>: &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; mtu <span class="hljs-number">1450</span> qdisc noqueue state UNKNOWN group <span class="hljs-keyword">default</span> 
    link/ether <span class="hljs-number">0</span>e:bc:ce:<span class="hljs-number">0</span>b:<span class="hljs-number">12</span>:<span class="hljs-number">1f</span> brd ff:ff:ff:ff:ff:ff
    inet <span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.0</span>/<span class="hljs-number">32</span> scope global flannel<span class="hljs-number">.1</span>
       valid_lft forever preferred_lft forever
    inet6 fe80::cbc:ceff:fe0b:<span class="hljs-number">121f</span>/<span class="hljs-number">64</span> scope link 
       valid_lft forever preferred_lft forever

# <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.212</span>
<span class="hljs-number">5</span>: docker0: &lt;NO-CARRIER,BROADCAST,MULTICAST,UP&gt; mtu <span class="hljs-number">1500</span> qdisc noqueue state DOWN group <span class="hljs-keyword">default</span> 
    link/ether <span class="hljs-number">02</span>:<span class="hljs-number">42</span>:db:e3:fc:<span class="hljs-number">2</span>a brd ff:ff:ff:ff:ff:ff
    inet <span class="hljs-number">10.10</span><span class="hljs-number">.88</span><span class="hljs-number">.1</span>/<span class="hljs-number">24</span> brd <span class="hljs-number">10.10</span><span class="hljs-number">.88</span><span class="hljs-number">.255</span> scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::<span class="hljs-number">42</span>:dbff:fee3:fc2a/<span class="hljs-number">64</span> scope link 
       valid_lft forever preferred_lft forever
<span class="hljs-number">6</span>: flannel<span class="hljs-number">.1</span>: &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; mtu <span class="hljs-number">1450</span> qdisc noqueue state UNKNOWN group <span class="hljs-keyword">default</span> 
    link/ether fe:<span class="hljs-number">78</span>:<span class="hljs-number">71</span>:<span class="hljs-number">7</span>b:<span class="hljs-number">17</span>:ee brd ff:ff:ff:ff:ff:ff
    inet <span class="hljs-number">10.10</span><span class="hljs-number">.88</span><span class="hljs-number">.0</span>/<span class="hljs-number">32</span> scope global flannel<span class="hljs-number">.1</span>
       valid_lft forever preferred_lft forever
    inet6 fe80::fc78:<span class="hljs-number">71f</span>f:fe7b:<span class="hljs-number">17</span>ee/<span class="hljs-number">64</span> scope link 
       valid_lft forever preferred_lft forever

</code></pre>
<blockquote>
<p>集群中的路由条目会多出一个 10.10.0.0/16</p>
</blockquote>
<pre><code class="hljs angelscript"># 查看路由条目，多出了子网的路由条目，可以由docker0来转发
<span class="hljs-string">[root@localhost ~]</span># ip r
<span class="hljs-keyword">default</span> via <span class="hljs-number">192.168</span><span class="hljs-number">.1</span><span class="hljs-number">.3</span> dev ens33 proto static metric <span class="hljs-number">100</span> 
<span class="hljs-number">10.10</span><span class="hljs-number">.0</span><span class="hljs-number">.0</span>/<span class="hljs-number">16</span> dev flannel<span class="hljs-number">.1</span> 
<span class="hljs-number">172.17</span><span class="hljs-number">.0</span><span class="hljs-number">.0</span>/<span class="hljs-number">16</span> dev docker0 proto kernel scope link src <span class="hljs-number">172.17</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span> 
<span class="hljs-number">192.168</span><span class="hljs-number">.1</span><span class="hljs-number">.0</span>/<span class="hljs-number">24</span> dev ens33 proto kernel scope link src <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.211</span> metric <span class="hljs-number">10</span></code></pre>





</li>
</ul>
</li>
</ul>
<blockquote>
<p>开启容器，两台主机的容器可以互相连通，flannel没有隔离性</p>
</blockquote>
<pre><code class="hljs angelscript"># <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.211</span>
<span class="hljs-string">[root@localhost ~]</span># docker run -itd --name bbox1 busybox
<span class="hljs-number">546</span>d5f6c62a84a4ff8e5fa670b926f7155dd1f6426f31495370cb949b4ad2bfb
d[<span class="hljs-symbol">root@</span>localhost ~]# docker exec -it bbox1 ip a
<span class="hljs-number">1</span>: lo: &lt;LOOPBACK,UP,LOWER_UP&gt; mtu <span class="hljs-number">65536</span> qdisc noqueue qlen <span class="hljs-number">1000</span>
    link/loopback <span class="hljs-number">00</span>:<span class="hljs-number">00</span>:<span class="hljs-number">00</span>:<span class="hljs-number">00</span>:<span class="hljs-number">00</span>:<span class="hljs-number">00</span> brd <span class="hljs-number">00</span>:<span class="hljs-number">00</span>:<span class="hljs-number">00</span>:<span class="hljs-number">00</span>:<span class="hljs-number">00</span>:<span class="hljs-number">00</span>
    inet <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>/<span class="hljs-number">8</span> scope host lo
       valid_lft forever preferred_lft forever
<span class="hljs-number">9</span>: <span class="hljs-symbol">eth0@</span>if10: &lt;BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN&gt; mtu <span class="hljs-number">1450</span> qdisc noqueue 
    link/ether <span class="hljs-number">02</span>:<span class="hljs-number">42</span>:<span class="hljs-number">0</span>a:<span class="hljs-number">0</span>a:<span class="hljs-number">55</span>:<span class="hljs-number">02</span> brd ff:ff:ff:ff:ff:ff
    inet <span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.2</span>/<span class="hljs-number">24</span> brd <span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.255</span> scope global eth0
       valid_lft forever preferred_lft forever

# <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.212</span>
<span class="hljs-string">[root@localhost ~]</span># docker run -itd --name bbox2 busybox
<span class="hljs-number">99</span>a6f6729ceda4bdd7483c4ad0cb831bbf24d715a7ce4af6fdf8bcd037d188c6
<span class="hljs-string">[root@localhost ~]</span># docker exec -it bbox2 ip a
<span class="hljs-number">1</span>: lo: &lt;LOOPBACK,UP,LOWER_UP&gt; mtu <span class="hljs-number">65536</span> qdisc noqueue qlen <span class="hljs-number">1000</span>
    link/loopback <span class="hljs-number">00</span>:<span class="hljs-number">00</span>:<span class="hljs-number">00</span>:<span class="hljs-number">00</span>:<span class="hljs-number">00</span>:<span class="hljs-number">00</span> brd <span class="hljs-number">00</span>:<span class="hljs-number">00</span>:<span class="hljs-number">00</span>:<span class="hljs-number">00</span>:<span class="hljs-number">00</span>:<span class="hljs-number">00</span>
    inet <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>/<span class="hljs-number">8</span> scope host lo
       valid_lft forever preferred_lft forever
<span class="hljs-number">9</span>: <span class="hljs-symbol">eth0@</span>if10: &lt;BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN&gt; mtu <span class="hljs-number">1450</span> qdisc noqueue 
    link/ether <span class="hljs-number">02</span>:<span class="hljs-number">42</span>:<span class="hljs-number">0</span>a:<span class="hljs-number">0</span>a:<span class="hljs-number">58</span>:<span class="hljs-number">02</span> brd ff:ff:ff:ff:ff:ff
    inet <span class="hljs-number">10.10</span><span class="hljs-number">.88</span><span class="hljs-number">.2</span>/<span class="hljs-number">24</span> brd <span class="hljs-number">10.10</span><span class="hljs-number">.88</span><span class="hljs-number">.255</span> scope global eth0
       valid_lft forever preferred_lft forever

# bbox1 分配了 <span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.0</span>/<span class="hljs-number">24</span>网段
# bbox2 分配了 <span class="hljs-number">10.10</span><span class="hljs-number">.88</span><span class="hljs-number">.0</span>/<span class="hljs-number">24</span>网段
</code></pre>
<ul>
<li>连通性测试<pre><code class="hljs routeros">[root@localhost ~]# docker exec -it bbox1 sh
/ #<span class="hljs-built_in"> ping </span>10.10.88.2
PING 10.10.88.2 (10.10.88.2): 56 data bytes
64 bytes <span class="hljs-keyword">from</span> 10.10.88.2: <span class="hljs-attribute">seq</span>=0 <span class="hljs-attribute">ttl</span>=62 <span class="hljs-attribute">time</span>=7.319 ms
64 bytes <span class="hljs-keyword">from</span> 10.10.88.2: <span class="hljs-attribute">seq</span>=1 <span class="hljs-attribute">ttl</span>=62 <span class="hljs-attribute">time</span>=0.344 ms
</code></pre>

</li>
</ul>
<h2 id="host-gw模式互通"><a href="#host-gw模式互通" class="headerlink" title="host-gw模式互通"></a>host-gw模式互通</h2><ul>
<li><p>该模式是通过路由转发的，找的是路由条目。不通过flannel.1封装udp转发</p>
</li>
<li><p>只需要基于以上实验，修改导入etcd的IP地址池那个文件中的类型即可</p>
<pre><code class="hljs routeros"> [root@localhost ~]# vim etcd.json
&#123;
  <span class="hljs-string">&quot;NetWork&quot;</span>:<span class="hljs-string">&quot;10.10.0.0/16&quot;</span>,
  <span class="hljs-string">&quot;SubnetLen&quot;</span>:24,
  <span class="hljs-string">&quot;Backend&quot;</span>:&#123;
    <span class="hljs-string">&quot;Type&quot;</span>:<span class="hljs-string">&quot;host-gw&quot;</span>
  &#125;
&#125;
<span class="hljs-comment"># 重新导入到etcd中</span>
[root@localhost ~]# etcdctl <span class="hljs-attribute">--endpoints</span>=http://192.168.100.211:2379 <span class="hljs-builtin-name">set</span> /usr/local/bin/network<span class="hljs-built_in">/config </span>&lt; /root/etcd.json
&#123;
  <span class="hljs-string">&quot;NetWork&quot;</span>:<span class="hljs-string">&quot;10.10.0.0/16&quot;</span>,
  <span class="hljs-string">&quot;SubnetLen&quot;</span>:24,
  <span class="hljs-string">&quot;Backend&quot;</span>:&#123;
    <span class="hljs-string">&quot;Type&quot;</span>:<span class="hljs-string">&quot;host-gw&quot;</span>
  &#125;
&#125;
</code></pre></li>
<li><p>另一台主机查看是否导入</p>
<pre><code class="hljs routeros">[root@localhost ~]# etcdctl <span class="hljs-builtin-name">get</span> /usr/local/bin/network/config
&#123;
  <span class="hljs-string">&quot;NetWork&quot;</span>:<span class="hljs-string">&quot;10.10.0.0/16&quot;</span>,
  <span class="hljs-string">&quot;SubnetLen&quot;</span>:24,
  <span class="hljs-string">&quot;Backend&quot;</span>:&#123;
    <span class="hljs-string">&quot;Type&quot;</span>:<span class="hljs-string">&quot;host-gw&quot;</span>
  &#125;
&#125;
</code></pre>
<blockquote>
<p>重启flanneld 查看mtu变成了1500</p>
</blockquote>
<pre><code class="hljs angelscript"> # <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.211</span>
<span class="hljs-string">[root@localhost ~]</span># systemctl restart flanneld
<span class="hljs-string">[root@localhost ~]</span># cat /run/flannel/subnet.env 
FLANNEL_NETWORK=<span class="hljs-number">10.10</span><span class="hljs-number">.0</span><span class="hljs-number">.0</span>/<span class="hljs-number">16</span>
FLANNEL_SUBNET=<span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.1</span>/<span class="hljs-number">24</span>
FLANNEL_MTU=<span class="hljs-number">1500</span>
FLANNEL_IPMASQ=<span class="hljs-literal">false</span>
# 查看路由条目
<span class="hljs-string">[root@localhost ~]</span># ip r
<span class="hljs-keyword">default</span> via <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.2</span> dev ens33 proto static metric <span class="hljs-number">100</span> 
<span class="hljs-number">10.10</span><span class="hljs-number">.0</span><span class="hljs-number">.0</span>/<span class="hljs-number">16</span> dev flannel<span class="hljs-number">.1</span> 
<span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.0</span>/<span class="hljs-number">24</span> dev docker0 proto kernel scope link src <span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.1</span> 
<span class="hljs-number">10.10</span><span class="hljs-number">.88</span><span class="hljs-number">.0</span>/<span class="hljs-number">24</span> via <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.212</span> dev ens33 
<span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.0</span>/<span class="hljs-number">24</span> dev ens33 proto kernel scope link src <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.211</span> metric <span class="hljs-number">100</span> 
<span class="hljs-number">192.168</span><span class="hljs-number">.122</span><span class="hljs-number">.0</span>/<span class="hljs-number">24</span> dev virbr0 proto kernel scope link src <span class="hljs-number">192.168</span><span class="hljs-number">.122</span><span class="hljs-number">.1</span> 

 # <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.212</span>
<span class="hljs-string">[root@localhost ~]</span># systemctl restart flanneld
<span class="hljs-string">[root@localhost ~]</span># cat /run/flannel/subnet.env 
FLANNEL_NETWORK=<span class="hljs-number">10.10</span><span class="hljs-number">.0</span><span class="hljs-number">.0</span>/<span class="hljs-number">16</span>
FLANNEL_SUBNET=<span class="hljs-number">10.10</span><span class="hljs-number">.88</span><span class="hljs-number">.1</span>/<span class="hljs-number">24</span>
FLANNEL_MTU=<span class="hljs-number">1500</span>
FLANNEL_IPMASQ=<span class="hljs-literal">false</span>
# 查看路由条目
<span class="hljs-string">[root@localhost ~]</span># ip r
<span class="hljs-keyword">default</span> via <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.2</span> dev ens33 proto static metric <span class="hljs-number">100</span> 
<span class="hljs-number">10.10</span><span class="hljs-number">.0</span><span class="hljs-number">.0</span>/<span class="hljs-number">16</span> dev flannel<span class="hljs-number">.1</span> 
<span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.0</span>/<span class="hljs-number">24</span> via <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.211</span> dev ens33 
<span class="hljs-number">10.10</span><span class="hljs-number">.88</span><span class="hljs-number">.0</span>/<span class="hljs-number">24</span> dev docker0 proto kernel scope link src <span class="hljs-number">10.10</span><span class="hljs-number">.88</span><span class="hljs-number">.1</span> 
<span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.0</span>/<span class="hljs-number">24</span> dev ens33 proto kernel scope link src <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.212</span> metric <span class="hljs-number">100</span> 
<span class="hljs-number">192.168</span><span class="hljs-number">.122</span><span class="hljs-number">.0</span>/<span class="hljs-number">24</span> dev virbr0 proto kernel scope link src <span class="hljs-number">192.168</span><span class="hljs-number">.122</span><span class="hljs-number">.1</span> 
</code></pre></li>
<li><p>多出了集群中非本机容器的ip子网网段，并且由集群中该容器的物理机的地址去转发</p>
<ul>
<li>10.10.88.0/24 via 192.168.100.212 dev ens33 </li>
<li>10.10.85.0/24 via 192.168.100.211 dev ens33 </li>
</ul>
</li>
<li><p>也就是说两个主机之间的容器要进行通信，现在由各自主机的ens33来转发</p>
<blockquote>
<p>修改docker启动项中的MTU值(两台)</p>
</blockquote>
<pre><code class="hljs autoit">[root<span class="hljs-symbol">@localhost</span> ~]<span class="hljs-meta"># vim /usr/lib/systemd/system/docker.service</span>
<span class="hljs-meta"># 修改--mtu=1500即可</span>
<span class="hljs-meta"># 重启docker</span>
[root<span class="hljs-symbol">@localhost</span> ~]<span class="hljs-meta"># systemctl daemon-reload</span>
[root<span class="hljs-symbol">@localhost</span> ~]<span class="hljs-meta"># systemctl restart docker</span>
</code></pre></li>
<li><p>测试连通</p>
<pre><code class="hljs angelscript"># <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.211</span>
<span class="hljs-string">[root@localhost ~]</span># docker run -itd --name bbox4 busybox
e0347c838d53f724470377e2d51aedf2b0d2ec1826bb53d7afb1946ccf3ffe58
#<span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.212</span>
<span class="hljs-string">[root@localhost ~]</span># docker run -itd --name bbox3 busybox
bf0c3587d2e7ecf58605115259a835c7f6df08285eff2ac70b8d62b42c2debf0
<span class="hljs-string">[root@localhost ~]</span># docker exec -it bbox3 sh
/ # ping <span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.2</span>
PING <span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.2</span> (<span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.2</span>): <span class="hljs-number">56</span> data bytes
<span class="hljs-number">64</span> bytes <span class="hljs-keyword">from</span> <span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.2</span>: seq=<span class="hljs-number">0</span> ttl=<span class="hljs-number">62</span> time=<span class="hljs-number">0.428</span> ms
<span class="hljs-number">64</span> bytes <span class="hljs-keyword">from</span> <span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.2</span>: seq=<span class="hljs-number">1</span> ttl=<span class="hljs-number">62</span> time=<span class="hljs-number">0.460</span> ms

/ # traceroute <span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.2</span>
traceroute to <span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.2</span> (<span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.2</span>), <span class="hljs-number">30</span> hops max, <span class="hljs-number">46</span> byte packets
 <span class="hljs-number">1</span>  loaclhost (<span class="hljs-number">10.10</span><span class="hljs-number">.88</span><span class="hljs-number">.1</span>)  <span class="hljs-number">0.010</span> ms  <span class="hljs-number">0.008</span> ms  <span class="hljs-number">0.005</span> ms
 <span class="hljs-number">2</span>  <span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.211</span> (<span class="hljs-number">192.168</span><span class="hljs-number">.100</span><span class="hljs-number">.211</span>)  <span class="hljs-number">0.398</span> ms  <span class="hljs-number">0.440</span> ms  <span class="hljs-number">0.328</span> ms
 <span class="hljs-number">3</span>  loaclhost (<span class="hljs-number">10.10</span><span class="hljs-number">.85</span><span class="hljs-number">.2</span>)  <span class="hljs-number">0.204</span> ms  <span class="hljs-number">0.393</span> ms  <span class="hljs-number">1.410</span> ms

# 这时的路由跟踪看出，两个容器是经过物理机的ens33的IP地址来转发的。</code></pre>
<h2 id="vxlan和host-gw的区别"><a href="#vxlan和host-gw的区别" class="headerlink" title="vxlan和host-gw的区别"></a>vxlan和host-gw的区别</h2></li>
<li><p>host-gw把每个物理机都配置为容器的子网网关，每个物理机都知道各自内的子网和如何转发。</p>
</li>
<li><p>vxlan是在物理机之间建立vxlan隧道，不同的物理机的容器都在vxlan的一个大网中。</p>
</li>
<li><p>两种虽然都使用不同的机制建立容器间的通信，但是对于容器则不需要任何改变。</p>
</li>
<li><p>由于vxlan需要对数据进行额外封装，性能较差。而相反host-gw的性能就相对较好</p>
</li>
<li><p>默认的节点间数据通信方式是UDP转发; flannel默认使用8285端口作为UDP封装报文的端口，host gw使用8472端口</p>
</li>
</ul>
   <!-- - 可以从图片看到， flannel会分配一个大的subnet，以此分配网段，每个容器的ip会从这个大的subnet进行划分网段

  - Docker0：数据从原容器发出后，经过主机的docker0网卡，再转发到flannel1的网卡通过 p2p网卡：udp封装、按路由投递(最长匹配原则，找到和本地路由最接近的ip进行upd转发)
  - 到另一台主机，先解udp封装，相反操作，
  - Flannel网卡只支持docker0网卡向容器内进行转发



<!-- 
 - 作用
   - flannel实现跨主机通信是通过 docker0进行通信的 (修改docker0的ip) ，由flannel分配的子网都是从手动指定的一个大的子网中划分出来分配的。
   - 主机安装flannel网络后，每个主机都会有一个flanneld客户端，作用是联系etcd分配子网(存放网络配置、已分配的 subnet、host 的 IP 等信息)
   - etcd 会存储flannel提交的数据(配置信息，网络关系等)，通过backend转发
   - etcd（与 consul 类似的 key-value 分布式数据库）

   - Flannel实质上是一种overlay的网络类型，表示的是在一个网络应用层当中，不依靠ip地址传递消息，而是采用一种映射机制，把ip地址映射到某个资源定位上，也就将tcp的数据包包装到另外一种数据包上进行传输，目前支持udp、Vxlan、vps等转发方式。 -->



<!-- ## flannel工作原理
 - 每个主机配置一个ip段，和子网个数 例如可以配置一个覆盖网络使用 10.100.0.0/16端 每个主机/24个子网，因此主机a可以接受10.100.5.0/24 主机b可以接受10.100.18.0/24的包，flannel使用etcd来维护分配的子网，实际的ip地址间的映射
 - 对于数据路径，flannel使用udp来封装ip数据包，转发到远程主机，

 - 

 - flanneld指定了ip地址池后 (10.10.0.0/16) 并指定每台主机以/24的子网掩码来划分子网，当启动了flanneld服务之后，集群中每台物理机会生成一个 flannel.1的网卡，且网段已经是flanneld自动分配好的网段，然后修改docker0的网段改为，flanneld分配的网段，并加入到路由条目中，来自该地址池中的ip都会通过flannel.1网卡来转发数据包。

 - 如果是host_gw模式，会将docker0与flannel.1通过docker启动项里面增加bip和mtu的方式，是的子网网段由docker0转发， -->
            </div>
            <hr>
            <div>
              <div class="post-metas mb-3">
                
                  <div class="post-meta mr-3">
                    <i class="iconfont icon-category"></i>
                    
                      <a class="hover-with-bg" href="/categories/docker/">docker</a>
                    
                  </div>
                
                
                  <div class="post-meta">
                    <i class="iconfont icon-tags"></i>
                    
                      <a class="hover-with-bg" href="/tags/docker/">docker</a>
                    
                  </div>
                
              </div>
              
                <p class="note note-warning">本博客所有文章是以学习为目的，如果有不对的地方可以一起交流沟通共同学习 邮箱:1248287831@qq.com！</p>
              
              
                <div class="post-prevnext row">
                  <article class="post-prev col-6">
                    
                    
                      <a href="/2020/12/14/docker-weave/">
                        <i class="iconfont icon-arrowleft"></i>
                        <span class="hidden-mobile">docker-weave网络</span>
                        <span class="visible-mobile">上一篇</span>
                      </a>
                    
                  </article>
                  <article class="post-next col-6">
                    
                    
                      <a href="/2020/12/10/docker-overylay%E7%BD%91%E7%BB%9C/">
                        <span class="hidden-mobile">docker-overylay</span>
                        <span class="visible-mobile">下一篇</span>
                        <i class="iconfont icon-arrowright"></i>
                      </a>
                    
                  </article>
                </div>
              
            </div>

            
          </article>
        </div>
      </div>
    </div>
    
      <div class="d-none d-lg-block col-lg-2 toc-container" id="toc-ctn">
        <div id="toc">
  <p class="toc-header"><i class="iconfont icon-list"></i>&nbsp;目录</p>
  <div id="tocbot"></div>
</div>

      </div>
    
  </div>
</div>

<!-- Custom -->


    
  </main>

  
    <a id="scroll-top-button" href="#" role="button">
      <i class="iconfont icon-arrowup" aria-hidden="true"></i>
    </a>
  

  
    <div class="modal fade" id="modalSearch" tabindex="-1" role="dialog" aria-labelledby="ModalLabel"
     aria-hidden="true">
  <div class="modal-dialog modal-dialog-scrollable modal-lg" role="document">
    <div class="modal-content">
      <div class="modal-header text-center">
        <h4 class="modal-title w-100 font-weight-bold">搜索</h4>
        <button type="button" id="local-search-close" class="close" data-dismiss="modal" aria-label="Close">
          <span aria-hidden="true">&times;</span>
        </button>
      </div>
      <div class="modal-body mx-3">
        <div class="md-form mb-5">
          <input type="text" id="local-search-input" class="form-control validate">
          <label data-error="x" data-success="v"
                 for="local-search-input">关键词</label>
        </div>
        <div class="list-group" id="local-search-result"></div>
      </div>
    </div>
  </div>
</div>
  

  

  <footer class="text-center mt-5 py-3">
  <div class="footer-content">
     <a href="https://space.bilibili.com/39165374" target="_blank" rel="nofollow noopener"><span>LMK</span></a> <i class="iconfont icon-love"></i> <a href="https://github.com/fluid-dev/hexo-theme-fluid" target="_blank" rel="nofollow noopener"><span>Fluid</span></a> 
  </div>
  
  <div class="statistics">
    
    

    
      
        <!-- 不蒜子统计PV -->
        <span id="busuanzi_container_site_pv" style="display: none">
            总访问量 
            <span id="busuanzi_value_site_pv"></span>
             次
          </span>
      
      
        <!-- 不蒜子统计UV -->
        <span id="busuanzi_container_site_uv" style="display: none">
            总访客数 
            <span id="busuanzi_value_site_uv"></span>
             人
          </span>
      
    
  </div>


  

  
</footer>

<!-- SCRIPTS -->
<script  src="https://cdn.staticfile.org/jquery/3.4.1/jquery.min.js" ></script>
<script  src="https://cdn.staticfile.org/twitter-bootstrap/4.4.1/js/bootstrap.min.js" ></script>
<script  src="/js/debouncer.js" ></script>
<script  src="/js/main.js" ></script>

<!-- Plugins -->


  
    <script  src="/js/lazyload.js" ></script>
  



  



  <script defer src="https://cdn.staticfile.org/clipboard.js/2.0.6/clipboard.min.js" ></script>
  <script  src="/js/clipboard-use.js" ></script>



  <script defer src="https://busuanzi.ibruce.info/busuanzi/2.3/busuanzi.pure.mini.js" ></script>





  <script  src="https://cdn.staticfile.org/tocbot/4.11.1/tocbot.min.js" ></script>
  <script>
    $(document).ready(function () {
      var boardCtn = $('#board-ctn');
      var boardTop = boardCtn.offset().top;

      tocbot.init({
        tocSelector: '#tocbot',
        contentSelector: '#post-body',
        headingSelector: 'h1,h2,h3,h4,h5,h6',
        linkClass: 'tocbot-link',
        activeLinkClass: 'tocbot-active-link',
        listClass: 'tocbot-list',
        isCollapsedClass: 'tocbot-is-collapsed',
        collapsibleClass: 'tocbot-is-collapsible',
        collapseDepth: 0,
        scrollSmooth: true,
        headingsOffset: -boardTop
      });
      if ($('.toc-list-item').length > 0) {
        $('#toc').css('visibility', 'visible');
      }
    });
  </script>



  <script  src="https://cdn.staticfile.org/typed.js/2.0.11/typed.min.js" ></script>
  <script>
    var typed = new Typed('#subtitle', {
      strings: [
        '  ',
        "docker-flannel&nbsp;",
      ],
      cursorChar: "_",
      typeSpeed: 70,
      loop: false,
    });
    typed.stop();
    $(document).ready(function () {
      $(".typed-cursor").addClass("h2");
      typed.start();
    });
  </script>



  <script  src="https://cdn.staticfile.org/anchor-js/4.2.2/anchor.min.js" ></script>
  <script>
    anchors.options = {
      placement: "right",
      visible: "hover",
      
    };
    var el = "h1,h2,h3,h4,h5,h6".split(",");
    var res = [];
    for (item of el) {
      res.push(".markdown-body > " + item)
    }
    anchors.add(res.join(", "))
  </script>



  <script  src="/js/local-search.js" ></script>
  <script>
    var path = "/local-search.xml";
    var inputArea = document.querySelector("#local-search-input");
    inputArea.onclick = function () {
      searchFunc(path, 'local-search-input', 'local-search-result');
      this.onclick = null
    }
  </script>



  <script  src="https://cdn.staticfile.org/fancybox/3.5.7/jquery.fancybox.min.js" ></script>
  <link  rel="stylesheet" href="https://cdn.staticfile.org/fancybox/3.5.7/jquery.fancybox.min.css" />

  <script>
    $('#post img:not(.no-zoom img, img[no-zoom]), img[zoom]').each(
      function () {
        var element = document.createElement('a');
        $(element).attr('data-fancybox', 'images');
        $(element).attr('href', $(this).attr('src'));
        $(this).wrap(element);
      }
    );
  </script>















</body>
</html>
